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Introductory Material - Definitions 
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Reliability Engineering 
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• Reliability engineering is a design function that involves the 
application of engineering principles to the design and 
processing of products, both hardware and software, for the 
purpose of meeting product reliability requirements or goals. 

• Reliability as a figure of merit is the probability that an item will 
perform its intended function for a specified mission profile. 
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Safety 
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• Safety 

o Safety is the freedom from those hazards that can cause death, 
injury, or illness in humans or adversely affect the environment. 

• System Safety (NASA/SP-2010-580, NASA System Safety Handbook) 

o System safety is the application of engineering and management 
principles, criteria, and techniques to optimize safety and reduce 
risks within the constraints of operational effectiveness, time, and 
cost throughout all phases of the system life cycle. 

o System safety to safety is as system engineering to engineering. 
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Risk Assessment 
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• Risk assessment is the process of determining the magnitude and consequences 
of risk. 

• Risk is the uncertainty regarding the future outcome of an undertaking of some 
kind, e.g., a decision alternative, a project, a launch, etc. 

• In the context of mission execution, risk is the expression of the potential for 
shortfalls related to any one or more of the mission execution domains (i. e. 
Safety, Technical performance, Cost, and Schedule). 

• Risk is defined as a set of triplets: 

o The scenario leading to a shortfall with respect to one or more Performance 
Measures (e.g., scenarios leading to injury, fatality, destruction of key assets; 
scenarios leading to exceedance of mass limits; scenarios leading to cost 
overruns; scenarios leading to schedule slippage) 

o The likelihood of the scenario (Quantitative/Qualitative). 

o The consequences (severity) of the performance degradation that would 
result if the scenario was to occur. 
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Risk Management 
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• In general, risk management is the systematic and iterative 
optimization of the project resources according to a risk 
management policy. 

o The risk management process consists of: 

■ Defining a risk management policy 

■ Identification and assessment of risks 

■ Determining risk mitigation and control 

■ Monitoring, communication, and acceptance of risks. 
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Risk Management 
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• NPR 8000. 4A, NASA Risk Management Procedural Requirements, involves 
two complementary processes: 

o Risk-informed Decision Making (RIDM) 

• To risk-inform direction-setting decisions (alternative selection). 

• To risk-inform the development of credible performance 
requirements as part of the overall systems engineering process. 

o Continuous Risk Management (CRM) 

• To manage risk associated with the implementation of baseline 
performance requirements. 


RM = RIDM + CRM 
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The RIDM Process 



• Identification of decision alternatives 
(decision context) and considering a 
sufficient number and diversity of 
Performance Measures. 


• Risk analysis of decision alternatives 
(uncertainty analysis of performance 
associated with the alternative). 


• Deliberation and Selection of a decision 
alternative informed by (not solely based 
on) Risk Analysis Results. 
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Risk-Informed Decision Making (RIDM) 



To Performance Requirements Development 

CRM 
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CRM is conducted in the context 
of performance requirements 


Steps in the CRM Process 
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CRM 

Feedback 










Introductory Material 
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Probability Basics 


11 


Probability 
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• The probability of an event A is a quantity that satisfies the following 
axioms (Kolmogorov, 1933): 

0 < Pr(A) < 1 

Pr(certain event) = 1 

• Imagine a large number (n) of repetitions of the "experiment" in which A 
is a possible outcome. 

• If A occurs k times, then its relative frequency is: k/n. 

• It is postulated that: 

lim k/n when n^oo is Pr(A). 

• In terms of Bayesian statistics, probability is defined as "Degree of Belief" 
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Probability Basic Rules 
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• The sum of the probabilities that an event will occur plus the probability 
that an event will not occur is equal to 1. 


P(A) + P(not A) = 1 


\ 



Example: The probability of rolling an even or 
an odd number on a fair die is 1. 


Probability Basic Rules 
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• If two events can't occur at the same time, they are called mutually 
exclusive events. The probability of either one event or the other event 
happening is equal to the sum of the probabilities of the individual 
events. 


P(XorY) = P(X)+P(Y) 
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Probability Basic Rules 



NSC 

NASA SAFETY CENTER 


• If two events can happen at the same time, the probability of either event 
happening is equal to the sum of the probabilities of the individual events 
minus the probability of both events occurring simultaneously. 

P(X or Y) = P(X) + P(Y) - P(X and Y) 

o If the two events are independent, the probability of both events 
occurring is equal to: 

P(X and Y) = P(X) * P(Y) 

o If the two events are dependent, the probability of both events 
happening consecutively is equal to: 

P(X and Y) = P(X) * P(Y/X) 

Where P(Y/X) is the conditional probability of Y given that X has already 
occurred. 
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Probability Related Functions 
(Using Exponential for Illustration) 
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Probability Density Function, f(t) 

• It describes where failures occur 
over time. 

• The area under curve is always = 1. 



Time 


Cumulative Density Function, F(t) 

• It displays the percentage of the 
population that has failed at some 
specific time "t" 

• It ranges between 0 and 1 and 
equals the area under the f(t) curve 
up to time "t" 
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Probability Related Functions 
(Using Exponential for Illustration) 



NSC 

NASA SAFETY CENTER 


Reliability Function, R(t) 

• For a given t, it represents the 
percentage that has survived to 
that point. 

• Is always between 0 and 1. 

• R(t)=l-F(t) 


Flazard Function 

• h(t) is the instantaneous failure 
rate at time t. 

. h(t)=f(t)/(l-F(t)) 


Other general relationships 



Tirr.e 


OO 


K(t) = e~ J- OO h (Odt MTTF = J R(t)dt 


0 


F. Safie 


17 



NSC 

NASA SAFETY CENTER 


Risk Assessment Tools and Techniques 
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The Risk Matrix 
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• Purpose: 

o Sort through a large amount of risks and determine which are most 
important. 

o Separate out which risks should be dealt with first (the vital few 
risks) when allocating resources. 

• Description: 

o Involves partitioning risks or groups of risks and ranking risks or sets 
of risks based upon a criterion or set of criteria, 
o Ranking should yield the project's "Primary Risks." 
o NPR 8000.4, Agency Risk Management Procedural Requirements, 
defines a Primary Risk as those undesirable events (risks) having 
both high probability and high impact/severity, 
o Characterization of a Primary Risk as "Acceptable" shall be 

supported by rationale, with concurrence of the Governing Program 
Management Council (GPMC), that all reasonable mitigation options 
within cost, schedule, and technical constraints have been 
instituted. 
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Standardized Agency 5x5 Risk Matrix 
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NOTE: Specific criteria for each of the Likelihood and Consequence categories are to be 
defined by each Enterprise or Program. Criteria may be different for manned missions, 
expendable launch vehicle missions, robotic missions, etc. 
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Risk Matrix is a Communication Tool 
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Risk communication: 

• Risk communication is the data exchange on risks necessary for decision 
making on a project. 

• Risk communication is supported by project reporting lines. 

• The type of decision determines the contents and format of risk data 
addressed to the decision-maker. 


Identification and Analysis 
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Risk Matrix 
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• Advantages 

o The risk matrix is a risk assessment communication tool. 

o The risk matrix provides a useful guide for prudent engineering. 

o The risk matrix provides a standard tool of treating the relationship 
between severity and probability in assessing risk for a given hazard. 

o Assessing risk subjectively avoids unknowingly accepting intolerable 
and senseless risk, allows operating decisions to be made, and 
improves resource distribution for mitigation of loss resources. 

• Limitations 

o The risk assessment matrix does not assist the analyst in identifying 
hazards. It can only be used if hazards are already identified and 
analyzed. 

o The risk assessment matrix is dominated by subjective information. 
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Probabilistic Risk Assessment (PRA) 
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• Probabilistic risk assessment is the systematic process of analyzing a 
system, a process, or an activity to answer three basic questions: 

o What can go wrong that would lead to loss or degraded performance 
(i.e., scenarios involving undesired consequences of interest)? 

o How likely is it (probabilities)? 

o What is the severity of the degradation (consequences)? 

• PRA is the task of generating the triplet set: 


R = RISK = {S„ P„ CJ 



Likelihood 
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Consequence 
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• Probabilistic risk assessment is a 
formal method to derive and quantify 
this set in an integrated manner. 

• This provides a framework to 
prioritize risks, identify risk 
contributors, and quantify cumulative 
(aggregate) risk and associated 
uncertainties. 
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The PRA Process 
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Probabilistic Risk Assessment (PRA) 

The Skills Needed 
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Probabilistic Risk Assessment (PRA) 
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• Advantages 

o Imposes logic structure on risk assessment. 

o Evaluates risk at various system levels including system interactions. 

o Handles multiple failures and common causes. 

o Provides more insight into the various system failure modes and the 
effects of human/process interaction. 

o Supports sensitivity analysis. 

o Provides a tool to combine both qualitative and quantitative risk 
analysis. 

o Can be useful in evaluating risk reduction, risk ranking, identifying 
areas that require further attention, and identifying system scenarios 
that have major impact on system risk. 

o PRA is a good source of data for sanity check of the likelihood input 
data of the risk matrix. 
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Probabilistic Risk Assessment (PRA) 
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• Limitations 

o Could be very expensive. 

o PRA faces a level of skepticism with respect to basic sources of 
quantification, basic failures/events modeled, basic quantification 
methods, completeness in covering all significant scenarios, 
quantification of uncertainty, etc. 

o It is very difficult to account for design margins, maturity, 

manufacturing capabilities and uncertainties, unexpected failure 
modes, unexpected common-cause failures, dependency, etc. 
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Fault Tree Analysis (FTA) 
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FTA is "An analytical technique, whereby an undesired state of the system is 
specified, and the system is then analyzed in the context of its environment 
and operation to find all credible ways in which the undesired event can 
occur." 


Fault Tree Handbook , NUREG-0492, 1981 
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Fault Tree Analysis (FTA) 
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• A fault tree analysis (FTA) is a top-down symbolic logic model generated in the 
failure domain. 

• This model traces the failure pathways from a predetermined, undesirable 
condition or event, called the TOP event, of a system to the failures or faults 
(fault tree initiators) that could act as causal agents. 

• An FTA can be carried out either quantitatively or qualitatively. 

• A quantitative FTA includes generating a fault tree (symbolic logic model), 
entering failure probabilities for each fault tree initiator, propagating failure 
probabilities to determine the TOP event failure probability, and determining 
cut sets that lead to the TOP event. 

• A cut set is any group of initiators that will, if they all occur, cause the TOP 
event to occur. 

• A minimal cut is a least group of initiators that will, if they all occur, cause the 
TOP event to occur. 

• FTAs are frequently used in the context of a larger model that analyzes a 
combination of events, each of which might be a top event of a separate fault 
tree. 


F. Safie 


29 


Fault Tree Analysis (FTA) 




• An FTA process involves the following steps: 
o Identify the objective for the FTA 
o Define the scope of the FTA 
o Define ground rules for the FTA 
o Define the resolution of the FTA 
o Define the top event of the FT 
o Construct the FT 
o Evaluate the FT 
o Interpret and present the results 
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Fault Tree Analysis (FTA) 
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Basic Fault Tree Logic 
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Fault Tree Analysis (FTA) 
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• Advantages 


o 

o 

o 

o 

o 


Enables assessment of probabilities of combined faults/failures 
within a complex system. 

Single-point failures can be identified and assessed. 

System vulnerability and low-payoff countermeasures are identified, 
thereby guiding deployment of resources for improved control of 
risk. 

Can be used to reconfigure a system to reduce vulnerability. 

Can be used in problem investigation. 
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Fault Tree Analysis (FTA) 
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• Limitations 


o 


o 


Addresses only one undesirable condition or event that must be 
foreseen by the analyst. Thus, several or many FTAs may be needed 
for a particular system. 

The generation of an accurate quantitative FTA may require 
significant time and resources. Caution must be taken not to "over 
work" determining probabilities or evaluating the system (i.e., limit 
the size of the tree). 

A fault tree is not accurate unless all significant contributors of faults 
or failures are anticipated. 
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Event Sequence Diagram (ESD) 
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• An ESD is essentially a flow chart with paths leading to different end 
states. Each path through the chart is a scenario. 

• The method uses forward logic. 

• Its primary use is supporting probabilistic risk assessments. 

• An ESD is developed for each initiating event category in the PRA Master 
Logic Diagram. 

• The input to an ESD is a defined initial state or "initiating event," the 
"pivotal events," and the end states of the scenarios of concern. 
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Event Sequence Diagram (ESD) 
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ESD Logic 
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Event Sequence Diagram (ESD) 
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• Advantages 

o ESD is useful for identifying accident scenarios. 

o Most engineers find ESDs intuitive and easy to understand. 

o Used for communication between PRA analysts and the engineering 
community. 




Limitations 

o Developing ESDs requires strong engineering knowledge and 
background in logic flows. 

o An ESD is qualitative in nature. 
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Event Tree Analysis (ETA) 
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• ETA is a forward binary logic modeling technique used to determine the 
propagation paths (sequence of events) to a set of final states resulting 
from a given initial state or condition. 

• In technical risk assessment, the initial state, or "initiating event," is 
generally a hazard or failure, the intermediate events are potential 
propagation paths, and the final states are either conditions of loss 
(degradation, damage, destruction, death) or successful loss mitigation. 

• Event tree analysis is generally applicable for almost any type of risk 
assessment application, but used most effectively to model accidents 
where multiple safeguards are in place as protective features. 

• ETA is a primary tool of Probabilistic Risk Assessment. 
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Event Tree Analysis (ETA) 
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ETA Logic 
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Event Tree Analysis (ETA) 
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• Advantages 

o Provides a clear order of events from the initial to end states, 
o Allows discrete time sequencing within failure scenarios, 
o Accounts for complexities of design mitigations (safety features, etc.). 

• Limitations 

o Can become exceedingly complex and require simplification, 
o Separate trees required for each initiating event. 

- Difficult to represent interactions among events 

- Difficult to consider effects of multiple initiating events 

o Requires ability to define a set of initiating events that will produce all 
important accident sequences. 

o Very resource intensive if applied to every hazard or failure in a 
design. 


F. Safie 


39 


Bayesian Analysis 
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Bayesian Inference 

• In probability and statistics, Bayes 1 theorem (alternatively Bayes' law or 
Bayes 1 rule) relates current to prior belief. It also relates current to prior 
evidence. 

• With the Bayesian interpretation of probability, the theorem expresses 
how a subjective degree of belief should rationally change to account for 
evidence. This is Bayesian inference, which is fundamental to Bayesian 
statistics. 

• Bayesian inference is a method of statistical inference in which Bayes' 
Rule is used to update the probability for a hypothesis as evidence is 
acquired. 
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Bayesian Analysis 
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Why Bayesian? 

• Bayesian approach is a viable approach, especially when data is scarce 
and models are complex. 

o It is not limited to observed data. 

o It allows wide variety of evidence to play role in inference. 

o It allows uncertainty to be characterized and propagated through risk 
model. 

o Modern Bayesian formulation exploits computer power and 

simulation and avoids past computer limitations and approximations. 
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Bayesian Analysis 
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Bayesian vs Classical Statistic 

• Classical statistics tries to make inference on the unknown parameters via 
sampling failure times and establishing confidence intervals for 
parameters and eventually life length distribution percentiles (A and B 
allowable). 

• In the Bayesian approach, probability is a quantification of degree of belief. 

• Bayesian statistics uses the notion that uncertainty about the parameters 
can be expressed via probability distributions called prior distributions. 

• The prior distribution is key to a successful Bayesian analysis. 

• The construction of the prior distribution depends on careful 
quantification of sound expert judgment for the problem at hand. 

• This process requires the use of domain experts for defensible 
implementation. 
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Bayesian Analysis 
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The Bayesian Process 

• The general Bayesian procedure is: 

o Begin with a probability model for the process of interest. 

o Specify a prior distribution for parameter(s) in this model, 

quantifying uncertainty, i.e., quantifying degree of belief about the 
possible parameter values. 

o Observe data. 

o Obtain the posterior (i.e., updated) distribution for the parameter(s) 
of interest. 

o Check validity of model. 
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Bayesian Analysis 
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• Advantages 

o Provides a way to explicitly introduce assumptions regarding prior 
knowledge or ignorance. 

o Can bring the engineering judgment applied in the prior out in the open. 

o If the prior information is encouraging, less new testing may be needed to 
confirm a desired MTBF at a given confidence. 

• Limitations 

o Prior information may not be accurate, generating misleading conclusions. 

o Customers may not accept validity of prior data or engineering judgments. 

o There is no one "correct way" of inputting prior information (choice of prior) 
and different approaches can give different results. 
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Risk Assessment Related Tools and Techniques 
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• Risk Assessment Related tools and Techniques 
o Hazard Analysis 

o Failure Modes and Effects Analysis/Critical Items List (FMEA/CIL) 
o Reliability Predictions 
o Reliability Demonstration 
o Ishikawa Chart (Cause and Effect Diagram) 
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Concluding Remarks 
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• The tools and techniques discussed here represent just a selected set of 
a bigger list that the risk assessment community use. 

• The application of these tools and techniques depends on the objective 
of the risk assessment and the applicability of the tool or technique to 
the situation. 

• Caution should be exercised when model selection within a 
tool/technique could significantly affect the overall results and 
conclusions. Sensitivity analysis is one option to explore the impact of 
model selection on the results. 
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